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The Beginning Teacher Evaluation Study,; Phase II,owas 
a research project on effective teaching behavior—whati teachers do 
that significantly affects what and how pupils learn. The purposes of 
Phase II were to (1) -develop an assessment system for measuring 
teacher and student behaviors and other factors which could influence 
^ach of them and their interrelationships and (2) -generate hypotheses 
about the interrelationships among teacher and pupil behavior? and 
related -factors. Subjects were U1 second grade and 54 fifth grade 
experienced teachers in eight school districts in California. The 
Reading and Mathematics Observation System (RAHOS) was one of two 
observational systems used in the study. With RAHOS, a trained 
observer can record continuously the events in a regular classroom in 
comprehensive detail and in real time. The system permits the 
observer to focus either on the teacher, a group of students, or a 
small number of target students, depending on the purpose of the 
observation. "Scores" from the observations were used as measures of 
teacher performance for the analyses done in the study. (BC) 
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Leading and Mathematics Observation System: 
^ Description and Measurement of 

Time Usage in the Classroom* . ' 

. Robert C, Calf ee ' 
Kathryn Hoover Calfee 
Stanford University 

The Reading and Mathematics Observation System, RAMOS, is designe^d 
for real time documentation of classroom activities of teachers, student 
groups, or Individual students. It is best suited to observation of 
classroom instruction in the areas of reading and mathematics at the" 
elementary level. With it, a trained observer" can record continuously the 
events in a regular classroom in comprehensive detail dnd in real time. 
The system permits the observer to focus either on the teacher, a group 
of students, or a small number of target, students, depending on the pur- 
pose to the observation. 

Once an observation interval (generally between 30°minutes and an 

hour) is initiated', the observer records the events that transpire as a 

series of "line," each line containing categorical entries describing the 

nature of the event. These" lines provide answers to such questions as: 

Who are "the students being observed ? . How many are there in 
the group? Where are they. located in the room? Which target 
students, if any, are in the group? 

Who are the adults in the classroom? How many adults are 
there? Where are they, and with what student groups are 
they associated? 



*Speech. presented at the American Educational Research Association Meeting 
in San Francisco, April 19-76. ' ' ' . 
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.What is going on at ^n^j ^given time? tfhat is each student 
group and adult doing? If engaged in an instructional 
activity, how many students and adults are involved, and 
where are they? 

.Who is doing the instructing? What is the nature of the 
instructor (teacher, aide, volunteer, tutor, etc.)? What 
role is the instructor playing (direct instruction, dis- 
cipline, class management, etc.)? 

What is the content of instruction ? What subject matter 
is being taught? What skills and activities are involved' 
What materials are being used? ^^hat kinds of feedback are 
available to the students? 



What is the respon se of the group to instruction? What 
kind of responses are required or expected of the students? 
What is the judged level of attention to instruction' 
Relative adequacy of performance? Amount of social inter- 
action between students? 

From the records obtained in this manner, several measures were 
extracted for the purposes of the BTES project. The primary aim of these 
measures is to describe the manner in which time is used in the classroom. 
The measures inc|Luded (1) total time spent on activities directly or 
indirectly related to reading and mathematics, (2) the character -.nd 
variety of thos^' activities, and (3) the relative distribution of time 
spent in variou^ activities. 

Procedures for Conducting a RAMOS Observation 
A RAMOS observer used several forms while observing in Phase II 
classrooms: a Start Form, an Event Form, a Debriefing Form, a Classroom 
Summary Record, and General Descriptive comments. The Event ^Form is 
most basic to the results described in this paper. 
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Figure 1 
about 
here 



Figure 2 
about 
here 



The Event Form. (Figure 1) is a labeled coding form used for recording 
each classroom event or change in real time. The initial condition of all 
existing groups is fully described, the instructor and the content of 
instruction is documented, and certain aspects of the student response are 
recorded. After the first lines have been completed, further lines are 
added as necessary to document any discernible changes in a group or in a 
target student within a group. Movement of students between groups or 
the formation of a new group also requires a line. 

Each event line records the time an event begins and indicate the 
group involved. Notations specific to instructors or target students are 
coded with the group association; Five major categories are recorded on 
each event line in RAMOS: Time, Group, Instructor, Content, and Response. 
Within each category, codes are entered in fields from the RAMOS Summary 
Codes sheet (Figure 2) . 

The fields for the TIME category are time an event begins, recorded. 
to the nearest minute on a 24-hour clock, and status, an indication of 
the focus of the observer. at this time. 

The GROUP category fields are the ID number indicating to which^ group 
the line applies, the number of students in the group, grid or relative 
position within space, location within room, and density of students in 
the group. 

The INSTRUCTOR category describes the activities of the instructors 
at a given time by the following fields: instructor ID shows' the specif i^ 

instructor, followed by his classification, the role he is now assuming^ 

/ 

his availability to students, how student-instructor interactions are 
being conducted ^ and mobility pf the instructor. « 
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The CONTENT category fields include the subject being studied, 
reading main and second skill, math main and second skill, main and 
secondary material, aim of instruction, activity used for instruction, 
feedback pattern of instructor, and feedback sign to students. 

The RESPONSE category fields indicate the task assigned to the 
group of students, the focus of attention of the students, their level of 
social interaction, output rate, quality of performance, level of atten- 
tion and involvement, physical activity, and noise level. There is also 
a column to record a code indicating when. a specific target student is 
referred to by this line. 

Analysis of RAMOS Protocols 
The basic unit of analysis for the RAMOS protocols is the student 
minute— a single number minute of student time during a class-period. 
The RAMOS analysis program computed, within aach category of the system, 
the number of student minutes falling within each of the codes assigned 
to that category. For instance, suppose that, during a given episode, 
one group of ten students spends 20 minutes reading, 5 minutes in class 
business; a second group of six students spends 12 minutes in mathematics, 
8 minutes in reading and 5 minutes in class business. Then under the 
SUBJECT category, the distribution of student minutes would be 248 for the 
Reading code ,(^0 x 10 for the first group, plus 6 x 8 for the second), 72 
for the Mathematics code. (6 x 12 for the second group) , and 80 for the 
Business code (10 x 5 for the first group, 6 x 5 for the second group). 
Of the total 400 minutes in the episode, the Reading code has the highest 
percentage, 62%, followed in second place by Business with 20% and 
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Mathematics with 18%. The analysis program computed the quantities- 
student minutes and percentages-for each code, and arranged the results 
in rank order from highest to. lowest. The series of observations for each 
of the 95 teachers were then cotubined into a single record for use in 
various analysis. Each teacher-class= «as observed on two to eight 
occasions. The data are averages over all observational occasions, so we 
are dealing with a single cu:Hulative observational record for' each teacher- 
class. 

The primary interest m. this study was to uncover any strong relations 
of teacher characteristics and practices to student outcomes in reading and 
mathematics. Accordingly, time usage in the areas of reading (and other 
closely allied language arts) and mathematics was chosen for analysis. 
Within these subject matter areas, four major categories were selected: 
role of the instructor, aim of instruction, instructional activities, and 

main materials. These categories answer the questions: Waat was the 
teacher doing (Role), to what purpose (Aims), by what means (Activities). " 
^. and with what resources (Materials)? These categories were chosen a priori 

as most likely to bear a relation to the overall pupil measures of reading 

and mathematics. 

The relative distribution of time usage by the target teacher (the 
"main instructor") in the variety of roles^ in the. RAMOS- data set is shown ' 
in Tables 1-6. ^ 

Table 1 shows the breakdown of time usage in the different sabject- 
matter codes. This table gives the proportion of teacher-class units in 
which the, code was observed at least once, the.mean percentage of student 
minutes, falling in that code, over alL teacher-class units, and- the mean 



Table 2 
about 
here. 



Table 3 

about 
" here 
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student minutes for the subject of teacher-class units in which the code 
was observed at least once. For example, in grade 1, the subject-matter 
code "Language" was observed in 70% of the classes, The n,ean percent of 
student minutes falling in that code over all classes was 8.7; the mean 
percent of student minutes falling in that code for only classes in which 
the code was observed was 12.4. 

The most noticeable result was the variation of direct instruction 
and facilitation as a function of grade level and subject matter (Table 2) . 
Direct instruction includes situations in which- the instructor is n,ain- 
taining direct control overlearning activity. Facilitation refers to 
activities in which the students work independently, but the instructor 
provides advice and feedback at the student's request or as a result of the 
instructor's direct observation. 

The results indicated that reading and language arts were much more 
likely to proceed by ^irect instruction, whereas mathematics was more likely 
to be under the studeAt's control with the teacher providing help when the 
student asks for it or\when the teacher sees something that requires atten- 
tion. Direct instruction is far more common in second grade than fifth 
grade, in both reading akd language arts and in mathematics. 

Management, evaluation, and other activities occupied about 25 percent 
of the instructor's time ider all conditions. ' Assessment and evaluation 
are more likely to take up t^me in observable ways In second grade than in 
fifth grade. \' 

The second major dimeas ion is related to the aim of instruction . ^ , 
(Table 3). The dominant aim,\ accounting for most of the time, is practicing- 
skills, either directly or through review of some sort . Relatively little 
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time was spent on new ideas or topics, and only in fifth grade language 
arts was any substantial amount cf time spent in the application of 
knowledge, skills on concepts to practical problems beyond a school context. 

The breakdown of time usage by grade and subject matter for the 
category of Instructional activities (Table 4) suggested that students 
speat most of their time at seatwork. Fifth graders ware observed to do" 
so much more than second graders, and seatwork was observed more frequently 
in mathematics than in reading. This pattern parallels the variation in 
facilitation as an instructional role. Discussions and question-answer 
episodes took up about 20 to 25 percent of the time. About 10 percent of 
the reading time in second grade was spent in oral reading recitation and 
aboue;Nthe same proportioiii'of, the mathematics time was spent in games of 
Nsome sort . 

The breakdown of time usage by grade and subject matter for the 
category Main materials (Table 5) suggested that there was large shifts 
in patterns of materials usage which were a function of grade level and 
subject matter. Main instructional materials were those that seemed of 
chief importance to the activity carried out by the student. If other 
materials also played a significant role, this was recorded as secondary 
material. Secondary material (Table 6) reflects the extensiveiiess of the 
instructional materials being^uped at the time in the classroom. For 
example, in mathematics instruction, it is the difference between a 
lesson in a workbook and a l^^son in which the student also uses a number 
scale to check his work. 4 example in reading is the difference between 
a lesson where the student reads to himself from a book and^one in which 
he records his thoughts and reactions in writing as he reads. 

9 
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There were large differences in the patterns of time usage as a 
function of grade level and subject matter. Books were used more in 
reading than in mathematics, and more at the later than the earlier 
grades — an expected result. Workbooks were coiranon everywhere, but especi- 
ally in second grade mathemtics. They also occupied a surprising amount 
of the time in fifth grade language arts and reading. About 15 to 20 
percent of the time, students were left on their own with paper and pencil. 
In second grade, students were working with no materials about 20 percent 
of the time; it seems likely that much'' of this time was accounted for by 
class discussion. Only in second grade mathematics was a considerable, 
amount of time spent with manipulative (Cuisenaire rods, number scales, 
etc.) or equipment (film strip, projectors, tape recorders, etc.). Generally, 
instruction in these classrooms was carried out with the old standbys — 
books, paper and pencils, and chalkboard. 

Two sets of correlations among various combinations of the RAMOS 
variables were especially noteworthy. One seems to encompass a pattern 
of direct instruction in which books and workbooks are used by students 
at their seats for practice and review purposes. The second pattern 
appears to represent situations in which teachers are more likely to spend 
their time in something other than direct instruction. Students are 
engaged with new.concepts and facts ^with unconventional materials and 
often in group discussion. More detailed examination of the data would be 
needed to confirm the existence of such patterns but they do seem to 
parallel descriptions of contrasting teaching styles found in contemporary 
classrooms. - 

10 
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These are the variables that were entered into the structural 
analysis. They suianarize a great amount of information, and undoubtedly 
some useful relationships have been obscured along the way. However, 
given the thesis that total time usage is one of the more fundamental 
-variables determining instructional effectiveness", these summary vari- 
ables represent a reasonable choice /or initial stages of hypotheses 
generation. 
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Basic Statistics for Time Usage 
in Subject-Matter Category 







% of classes 


Mean %. Student 
' Minutes 


— - 


LiOUQ 
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Given code 
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